Digital data processing apparatus having asymmetric hardware multithreading support for different threads

ABSTRACT

Asymmetric hardware support for a special class of threads is provided. Preferably, the special class threads are high-priority, I/O bound threads. In a first aspect, a multithreaded processor contains N sets of registers for supporting concurrent execution of N threads. At least one of the register sets is dedicated for use by a special class of threads, and can not be used by other threads even if idle. In a second aspect, the special class of threads can fill only the a limited portion of the cache memory, in order to reduce flushing of the cache which might otherwise occur.

FIELD OF THE INVENTION

The present invention relates to digital data processing hardware, andin particular to the design and operation of digital data processinghardware for supporting the concurrent execution of multiple threadswithin a processing unit of a digital data processing device.

BACKGROUND OF THE INVENTION

In the latter half of the twentieth century, there began a phenomenonknown as the information revolution. While the information revolution isa historical development broader in scope than any one event or machine,no single device has come to represent the information revolution morethan the digital electronic computer. The development of computersystems has surely been a revolution. Each year, computer systems growfaster, store more data, and provide more applications to their users.

A modern computer system typically comprises a central processing unit(CPU) and supporting hardware necessary to store, retrieve and transferinformation, such as communications buses and memory. It also includeshardware necessary to communicate with the outside world, such asinput/output controllers or storage controllers, and devices attachedthereto such as keyboards, monitors, tape drives, disk drives,communication lines coupled to a network, etc. The CPU is the heart ofthe system. It executes the instructions which comprise a computerprogram and directs the operation of the other system components.

From the standpoint of the computer's hardware, most systems operate infundamentally the same manner. Processors are capable of performing alimited set of very simple operations, such as arithmetic, logicalcomparisons, and movement of data from one location to another. But eachoperation is performed very quickly. Programs which direct a computer toperform massive numbers of these simple operations give the illusionthat the computer is doing something sophisticated. What is perceived bythe user as a new or improved capability of a computer system is madepossible by performing essentially the same set of very simpleoperations, but doing it much faster. Therefore continuing improvementsto computer systems require that these systems be made ever faster.

The overall speed of a computer system (also called the “throughput”)may be crudely measured as the number of operations performed per unitof time. Conceptually, the simplest of all possible improvements tosystem speed is to increase the clock speeds of the various components,and particularly the clock speed of the processor. E.g., if everythingruns twice as fast but otherwise works in exactly the same manner, thesystem will perform a given task in half the time. Early computerprocessors, which were constructed from many discrete components, weresusceptible to significant clock speed improvements by shrinking andcombining components, eventually packaging the entire processor as anintegrated circuit on a single chip, and increased clock speed throughfurther size reduction and other improvements continues to be a goal.

In addition to increasing clock speeds, it is possible to increase thethroughput of an individual CPU or a system by increasing the averagenumber of operations executed per clock cycle. Modern computer systemsare designed to perform many operations concurrently, in order toincrease the average number of operations executed in a given time.Parallelism of various types is a common technique for boosting systemthroughput. For example, the reduced size and cost of individualprocessors has made it feasible, indeed common, to provide multiple CPUsoperating in parallel in a single computer system.

One particular form of parallelism in computer design is the use ofhardware multithreading within a computer processor. The term“multithreading”, as used in the context of processor design, is not thesame as the software use of the term, and for this reason the phrase“hardware multithreading” is often used to distinguish multithreading inthe context of processor design from “multithreading” or“multiprogramming” in the context of software. The software use of theterm means that a single process or task is subdivided into multiplerelated threads, which are capable of being dispatched independently forexecution. Hardware multithreading involves the concurrent execution ofmultiple software threads within a single processor. These threads mayrepresent completely independent tasks which are unrelated to oneanother. As used herein, the term “multithreading” refers to hardwaremultithreading, unless otherwise qualified.

A processor which supports hardware multithreading can support multipleactive threads at any instant in time. I.e, the dispatcher in theoperating system can dispatch multiple threads to the same processorconcurrently. From the perspective of the operating system, it appearsthat there are two processors, each executing a respective thread. Thereare multiple approaches to hardware multithreading. In a moretraditional form, sometimes called “fine-grained multithreading”, theprocessor executes N threads concurrently by interleaving execution on acycle-by-cycle basis. This creates a gap between the execution of eachinstruction within a single thread, which tends to reduce the effect ofwaiting for certain short term latency events, such as waiting for apipeline operation to complete. A second form of multithreading,sometimes called “coarse-grained multithreading”, multiple instructionsof a single thread are executed exclusively until the processorencounters some longer term latency event, such as a cache miss, atwhich point the processor switches to another thread. In a third form ofmultithreading, herein referred to as “dynamic multithreading”, aninstruction unit in the processor selects one or more instructions fromamong multiple threads for execution in each cycle according to currentprocessor and thread state.

Regardless of the type of hardware multithreading employed, all hardwaremultithreading tends to increase the productive utilization of certainprocessor resources because one or more active threads can exploitprocessor resources to execute instructions even while other threads arestalled, as for example, when waiting for a cache line to be filled.I.e., in a processor which supports only a single thread, someprocessing resource, such as a pipeline, may have to wait idle on any ofnumerous latency events. However, if multiple threads are active in theprocessor, the probability that the resource can be utilized inincreased. Put another way, a multithreaded processor increases theaverage number of operations executed per clock cycle in comparison to asimilar processor which supports only a single thread.

Typically, hardware multithreading involves replicating certainprocessor registers for each thread in order to independently maintainthe states of multiple threads. For example, for a processorimplementing a PowerPC™ archticture to perform multithreading, theprocessor must maintain N states to run N threads. Accordingly, thefollowing are replicated N times: general purpose registers, floatingpoint registers, condition registers, floating point status and controlregister, count register, link register, exception register,save/restore registers and special purpose registers. Additionally,certain special buffers, such as a segment lookaside buffer, can bereplicated or each entry can be tagged with a thread number. Also, somebranch prediction mechanisms, such as the correlation register and thereturn stack, should also be replicated. However, larger hardwarestructures such as caches and execution units are typically notreplicated, and are shared by all threads.

Thus, it can be seen that hardware multithreading involves replicationof hardware in the form of additional registers and other structuresneeded to maintain state information. While the number of threadssupported can vary, each thread requires additional hardware resourcewhich must be justified by the increase in utilization of the sharedhardware resources, such as execution units. The marginal improvement inutilization declines as more threads are added and the shared hardwareresources become more fully utilized, while the cost of each additionalthread is relatively constant. Therefore the number of threads supportedin most hardware multithreading processors is relatively small, with twobeing a common number.

In many system architectures, certain threads representing interruptsand other special processes run at a high priority. A particular exampleis I/O bound threads, i.e., threads which service I/O processes.Generally, these threads spend most of their time in a wait statewaiting for I/O completion, and when executing, execute often, but onlybriefly and do not require large hardware resource. When such a threadis waiting on an event and the event occurs, the operating systemdispatcher often dispatches the thread immediately to a processor (dueto its high priority), causing some currently executing thread to bepre-empted.

Although each I/O bound thread may execute only briefly when dispatched,the cumulative effect of numerous high-priority pre-emptions can reducethe efficiency of system operation. There is some overhead involved inpre-empting a currently executing thread, saving its state, anddispatching the I/O bound thread to the processor, and multiplied bymany such events this becomes significant additional work. Additionally,a high priority thread has a tendency to flush the contents of cache,even when executing only briefly. I.e., it will fill the cache, andparticularly the high-level cache nearest the processor, with data itrequires, resulting in the removal of data needed by other threads.

It is desirable to find improved techniques for processor operation anddesign which will avoid or mitigate some of the undesirable side effectsof servicing such high-priority threads.

SUMMARY OF THE INVENTION

In a first aspect of the preferred embodiment of the present invention,a processor which supports multithreading contains N sets of registersfor supporting the maintenance of thread state and concurrent executionof N threads. At least one of the register sets is dedicated for use bya special class of threads, and can not be used by threads of anothertype even if there is no thread of the special class executing.

In a second aspect of the preferred embodiment of the present invention,a special class of threads is limited in its use of cacne memory.Threads of the special class can fill only the a limited portion of thecache memory, in order to reduce flushing of the cache which mightotherwise occur.

In the preferred embodiment, a processor supports three sets ofregisters, of which one is dedicated for use by the special class ofthreads. Preferably, this special class comprises high-priority threadswhich typically execute only briefly, and in particular includes I/Obound threads. In general, it is expected that the dedicated registerset will be idle most of the time. When a special class thread isdispatched, its state is maintained in the dedicated register set. Thespecial class thread is further limited in the number of cache linesthat it can fill in the L1 I-cache and L1 D-cache. Preferably, thecaches are N-way associative, and some limited number of cache lines ineach associativity set is available for use by the special class thread.The special class thread can load data only to this limited portion ofcache, preventing it from flushing the full contents of cache.

In the preferred embodiment, the processor operates as a dynamicmultithreaded processor in which the instruction unit selectsinstructions for execution based on thread priority, the special classof threads having the highest priority. I.e., the instruction unitselects instructions for execution from the highest priority thread, tothe extent there are instructions available for immediate execution, andselects instructions from the lower priority threads to the extenthigher priority thread instructions are not immediately ready toexecute. However, alternative forms of multithreading could be used.

Preferably, the operating system's dispatcher dispatches threadsaccording to membership in the special class, indicating to theprocessor whether or not a dispatched thread is to execute in thespecial class register set.

Although it might be supposed that restricting one of N register sets toa certain class of threads, resulting in the register set being inactivemost of the time, would be less efficient than allowing any thread toexecute in any register set of N sets (increasing the utilization of theregister sets), this is not necessarily the case. As the number ofregister sets N grows, the lower priority register sets execute withless frequency, so that the benefit of an Nth set becomes problematical.By establishing a special register set for certain high-prioritythreads, which generally execute often, but only briefly, a dispatchedspecial thread typically displaces an inactive special thread in theprocessor's registers. There is no need to swap out an active,general-purpose thread, and then swap it back in when the special threadrelinquishes the processor. This reduces the volume of context changeactivity in the operating system suffered by the general purposethreads. This reduction in context change workload can more than offsetthe fact that the special register set is idle most of the time.Furthermore, by limiting the number of cache lines available to thespecial threads (which generally don't need very many cache linesanyway), these threads are effectively prevented from flushing the cacheduring their brief periods of execution, thus reducing disruption toother active threads.

The details of the present invention, both as to its structure andoperation, can best be understood in reference to the accompanyingdrawings, in which like reference numerals refer to like parts, and inwhich:

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is a high-level block diagram of the major hardware components ofa computer system utilizing asymmetric hardware multithreading supportfor different threads, according to the preferred embodiment of thepresent invention.

FIG. 2 represents in greater detail a hierarchy CPUs and various levelsof cache for temporarily storing portions of main memory, according tothe preferred embodiment.

FIG. 3 is a high-level diagram of the major components of a CPUincluding certain associated cache structures, according to thepreferred embodiment.

FIG. 4 is a representation of the general structure of a cache includingassociated accessing mechanisms, according to the preferred embodiment.

FIG. 5 is a conceptual illustration showing certain major softwarecomponents of memory 102 and the division of hardware and softwarefunction at different levels of abstraction in a computer system,according to the preferred embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to the Drawing, wherein like numbers denote like partsthroughout the several views, FIG. 1 is a high-level representation ofthe major hardware components of a computer system 100 which utilizesasymmetric hardware multithreading support for different threads,according to the preferred embodiment of the present invention. Themajor components of computer system 100 include one or more centralprocessing units (CPU) 101A-101D, main memory 102, cache memory 106,terminal interface 111, storage interface 112, I/O device interface 113,and communications/network interfaces 114, all of which are coupled forinter-component communication via buses 103, 104 and bus interface 105.

System 100 contains one or more general-purpose programmable centralprocessing units (CPUs) 101A-101D, herein generically referred to asfeature 101. In the preferred embodiment, system 100 contains multipleprocessors typical of a relatively large system; however, system 100could alternatively be a single CPU system. Each processor 101concurrently executes instructions stored in memory 102 on behalf ofmultiple active threads. Instructions and other data are loaded intocache memory 106 from main memory 102 for processing. Main memory 102 isa random-access semiconductor memory for storing data, includingprograms. Although main memory 102 and cache 106 are representedconceptually in FIG. 1 as single entities, it will be understood that infact these are more complex, and in particular, that cache exists atmultiple different levels, as described in greater detail herein.Likewise, while some levels of cache may be shared among multiple CPUsas depicted in FIG. 1, other levels of cache may comprise multiple cachearrays, each closely coupled to a single CPU.

Buses 103-105 provide communication paths among the various systemcomponents. Memory bus 103 provides a data communication path fortransferring data among CPUs 101 and caches 106, main memory 102 and I/Obus interface unit 105. I/O bus interface 105 is further coupled tosystem I/O bus 104 for transferring data to and from various I/O units.I/O bus interface 105 communicates with multiple I/O interface units111-114, which are also known as I/O processors (IOPs) or I/O adapters(IOAs), through system I/O bus 104. System I/O bus may be, e.g., anindustry standard PCI bus, or any other appropriate bus technology.

I/O interface units 111-114 support communication with a variety ofstorage and I/O devices. For example, terminal interface unit 111supports the attachment of one or more user terminals 121-124. Storageinterface unit 112 supports the attachment of one or more direct accessstorage devices (DASD) 125-127 (which are typically rotating magneticdisk drive storage devices, although they could alternatively be otherdevices, including arrays of disk drives configured to appear as asingle large storage device to a host). I/O and other device interface113 provides an interface to any of various other input/output devicesor devices of other types. Two such devices, printer 128 and fax machine129, are shown in the exemplary embodiment of FIG. 1, it beingunderstood that many other such devices may exist, which may be ofdiffering types. Network interface 114 provides one or morecommunications paths from system 100 to other digital devices andcomputer systems; such paths may include, e.g., one or more networks 130such as the Internet, local area networks, or other networks, or mayinclude remote device communication lines, wireless connections, and soforth.

It should be understood that FIG. 1 is intended to depict therepresentative major components of system 100 at a high level, thatindividual components may have greater complexity that represented inFIG. 1, that components other than or in addition to those shown in FIG.1 may be present, and that the number, type and configuration of suchcomponents may vary. It will further be understood that not allcomponents shown in FIG. 1 may be present in a particular computersystem. Several particular examples of such additional complexity oradditional variations are disclosed herein, it being understood thatthese are by way of example only and are not necessarily the only suchvariations.

Although main memory 102 is shown in FIG. 1 as a single monolithicentity, memory may further be distributed and associated with differentCPUs or sets of CPUs, as is known in any of various so-callednon-uniform memory access (NUMA) computer architectures. Although memorybus 103 is shown in FIG. 1 as a relatively simple, single bus structureproviding a direct communication path among cache 106, main memory 102and I/O bus interface 105, in fact memory bus 103 may comprise multipledifferent buses or communication paths, which maybe arranged in any ofvarious forms, such as point-to-point links in hierarchical, star or webconfigurations, multiple hierarchical buses, parallel and redundantpaths, etc. Furthermore, while I/O bus interface 105 and I/O bus 104 areshown as single respective units, system 100 may in fact containmultiple I/O bus interface units 105 and/or multiple I/O buses 104.While multiple I/O interface units are shown which separate a system I/Obus 104 from various communications paths running to the various I/Odevices, it would alternatively be possible to connect some or all ofthe I/O devices directly to one or more system I/O buses.

Computer system 100 depicted in FIG. 1 has multiple attached terminals121-124, such as might be typical of a multi-user “mainframe” computersystem. Typically, in such a case the actual number of attached devicesis greater than those shown in FIG. 1, although the present invention isnot limited to systems of any particular size. Computer system 100 mayalternatively be a single-user system, typically containing only asingle user display and keyboard input, or might be a server or similardevice which has little or no direct user interface, but receivesrequests from other computer systems (clients).

While various system components have been described and shown at a highlevel, it should be understood that a typical computer system containsmany other components not shown, which are not essential to anunderstanding of the present invention.

FIG. 2 represents in greater detail a hierarchy CPUs and various levelsof cache for temporarily storing portions of main memory 102, accordingto the preferred embodiment. In this embodiment, there is a hierarchy ofcaches at levels designated level 1 (the highest level), level 2, andlevel 3, in addition to main memory. In general, caches become faster,and store progressively less data, at the higher levels (closer to theprocessor). Each processor 101 is associated with a respective pair oflevel 1 caches, which is not shared with any other processor. One cacheof this pair is a level 1 instruction cache (L1 I-cache) 201A, 201B(herein generically referred to as feature 201) for storing instructionsfor execution on the associated processor 101. The other cache of thepair is a level 1 data cache (L1 D-cache) 202A, 202B (herein genericallyreferred to as feature 202) for storing data (other than instructions)to be processed by threads executing on the associated processor 101.Each processor is further associated with a respective level 2 cache 203and a level 3 cache 204, which can generally store any form of data frommemory 102. Each pair of Level 1 caches 201, 202 is preferably dedicatedto a single corresponding processor; each L2 cache 203 and each L3 cache204 is typically shared among multiple processors, although one or moreof such caches could alternatively be dedicated to single respectiveprocessors. For clarity of illustration, FIG. 2 shows only twoprocessors 101, two pairs of L1 caches 210, 202, one L2 cache 203 andone L3 cache 204, but the number of processors and caches at variouslevels of system 100 could vary, and the number of processors sharing acache at each of the various levels could also vary. It should furtherbe understood that the number of cache levels could vary and that the L1caches needn't be divided into separate instruction and data caches, andthere could be more or fewer than three levels of cache.

FIG. 2 is intended to depict certain functional relationships among thevarious caches, and the fact that certain components are shownseparately is not intended as a representation of how the components arepackaged. Modern integrated circuit technology has advanced to the pointwhere at least some cache is typically packaged on the same integratedcircuit chip as a processor (sometimes also referred to as a processorcore, to distinguish it from caches and other mechanisms). Typically,the L1 caches 201, 202 are packaged on the same integrated circuit chipas the processor, and may be considered part of the processor. It iseven possible to place multiple processor cores on a single chip, with acertain amount of shared cache. For example, in the embodimentrepresented in FIG. 2, a pair of CPUs 101A, 101B, together with a pairof respective L1 I-caches 201A, 201B, a pair of respective L1 D-caches202A, 202B, and a single L2 cache 203 which is shared by both processors101A, 101B, are packaged on a single integrated circuit chip 210. The L3cache 204 is typically a separate integrated circuit chip or chipsmounted on a common circuit package along with the correspondingprocessor chip. However, this arrangement is only one possible packagingarrangement, and as integrated circuit and other electronics packagingtechnology evolves it is conceivable that further integration will beemployed.

Chip 210 includes a bi-directional external data communicationsinterface 205 for handling data communications with system componentsexternal to chip 210. Data paths exists between main memory 102, L3cache 204, and chip external data interface 205, and between interface205, between L2 cache 203 and L1 caches 201, 202. Although FIG. 2represents data paths conceptually as direct paths, they may in fact beimplemented as multiple successive buses or similar data paths.

FIG. 3 is a high-level diagram of the major components of CPU 101including certain associated cache structures, according to thepreferred embodiment, showing CPU 101 in greater detail than is depictedin FIGS. 1 or 2. CPU 101 includes instruction unit portion 301 andexecution unit portion 311. In general, instruction unit 301 obtainsinstructions from L1 I-cache 201, selects and decodes instructions todispatch to the execution unit, and resolves branch conditions tocontrol program flow. Execution unit 311 performs arithmetic and logicaloperations on data in registers, and loads or stores data from L1D-Cache 202. CPU 101 further includes address translation unit 322.

In this embodiment, CPU supports the concurrent execution withinprocessor 101 of up to three independent threads, it being understoodthat the number of threads supported could vary. Specifically, CPUmaintains state of up to three active threads, any of which may utilizehardware resources of the processor. In order to maintain state of threeactive threads, certain registers or portions thereof which maintainstate information are replicated for each thread. The active threads areherein designated threads 0, 1 and 2 (T0, T1 and T2), where T0 is thehighest priority thread and T2 the lowest. Thread T0 is reserved for useby threads of a special class, which are typically I/O bound threads andcertain other threads. Threads of this special class generally spendmost of their time in a wait state in the operating system, waiting onsome event (such as an I/O interrupt). When dispatched to a processorfor execution, they should execute at a high priority. Typically,execution is brief, and the thread then returns to the wait state. Theoperating system's dispatcher indicates whether a dispatched threadshould execute as thread T0 (a special thread) or as one of the othergeneral threads.

Instruction unit 301 comprises branch unit 302, instructiondecode/dispatch unit 303, and a set of instruction registers (also knownas instruction buffers) 304. Instructions from L1 I-cache 201 are loadedinto buffers 304 prior to execution. In the preferred embodiment, theinstruction unit includes three subsets of instruction buffers 305-307,each subset being the same size and corresponding to a respective threadsupported by the processor. Decode/dispatch unit 203 selects one or moreinstructions to be dispatched for execution from buffers 204 in acurrent machine cycle, and decodes the instruction(s) to determine theoperation(s) to be performed or branch conditions. Branch unit 202controls the program flow by evaluating branch conditions, and refillsbuffers 204 from L1 I-cache 221.

Execution unit 211 comprises multiple execution pipelines 312-314. FIG.3 represents a load-store pipeline 312, an integer arithmetic/logicpipeline 313, and a floating point pipeline 314. Although threerepresentative pipelines are illustrated in FIG. 3, it should beunderstood that the number of pipelines may vary, and that there may bemultiple pipelines of the same type. Each pipeline performs a definedoperation or set of operations on data in multiple stages correspondingto machine cycles. As is known in the art, data for an operation isintroduced into the pipeline in a first machine cycle, and advances onestage in the pipeline in each succeeding cycle. Provided that a pipelineis always full, it can perform an average of one operation per cycle, nomatter how many stages it takes to complete the operation (although infact it is impossible to keep the pipeline full at all times).

Execution unit 211 further includes a set of general purpose dataregisters 215 for storing data to be processed in pipelines 312-314according to instructions from instruction unit 301. The general purposedata registers include three subsets 316-318, each subset being the samesize and corresponding to a respective thread supported by theprocessor. For clarity of illustration, each subset 316-318 of dataregisters is represented as a single collection of registers of equalsize. However, in many processor architectures, data registers include aset of integer data registers and a separate set of floating point dataregisters, the latter often being longer. Where the processorarchitecture supports both integer and floating point registers, eachsubset 316-318 includes the full complement of integer and floatingpoint registers. Where the processor implementation includes moreregisters than the processor architecture, extras may be assigned to oneparticular subset 316-318 or shared among them.

In addition to components shown in FIG. 2, processor 101 may include anyof various special-purpose registers, such as machine state registers,condition registers, status registers and so forth (not shown). Wherespecial purpose registers contain state information specific to athread, this thread-specific information is replicated for each of thethreads supported by the processor.

Address translation unit 321 translates addresses generated byinstruction unit 201 or execution unit 211 (typically referred to as“virtual addresses” or “effective addresses”) to corresponding addressesin memory 102 (typically referred to as “real addresses” or “physicaladdresses”). Address translation unit preferably includes translationlook-aside buffer (TLB) 322, which is effectively a cache of certainaddress translation data from a page table in memory, and may includeother data as well. Depending on the architecture, address translationmay be dependent on the thread which generated the address. If this isthe case, each TLB entry contains a thread identifier which identifiesthe thread to which the TLB entry corresponds.

In the preferred embodiment, L2 cache 203 and L3 cache 204 are addressedusing real addresses, while L1 caches 201, 202 are addressed usingeffective addresses. The use of real addresses to access cachenecessarily means that addresses for data access must be translated byaddress translation unit 321 before accessing cache, but has significantadvantages in other respects, particularly in maintaining cachecoherency where a system has multiple processors and caches. Accessingcaches using real vs. effective addresses is an architectural designchoice involving many factors, and some system architectures useexclusively real addresses to access all caches, some use exclusivelyeffective addresses, and some use effective addresses for certain caches(higher-level caches) and real addresses for others. Although aparticular embodiment is described herein, the present invention is notlimited to any particular architecture, and could be employed in systemsusing any of these variations.

In operation, decode/dispatch unit 303 selects instructions fromregisters 304 for dispatch to execution unit 311. Each instructionexecuted in CPU 101 performs a single primitive operation, such as aload, a store, an integer arithmetic or logical operation, a floatingpoint operation, a branch, etc. Decode/dispatch unit 303 cansimultaneously dispatch multiple such instructions in a single machinecycle, and these instructions may be from the same thread or fromdifferent threads. Dispatch is the initial stage in processing theinstruction. In the case of instructions dispatched to a pipeline312-314 of the execution unit, the instruction may take multiple machinecycles to complete, and other instructions may be dispatched to the samepipeline in subsequent cycles.

Decode/dispatch unit 303 selects instructions for dispatch based onthread priority. If a hardware processing resource, such as a pipeline,is available to receive an instruction, and there is an availableinstruction in the T0 buffers 305 which is ready to use the processingresource, this instruction from the T0 buffers will be selected anddispatched. If the processing resource is available, but there is noavailable instruction in the T0 buffers which can use that resource, thedecode dispatch unit will select an available instruction from the T1buffers 306, if such an instruction exists, and failing that, select anavailable instruction from the T2 buffers 307, if such an instructionexists. A processing resource is not always available to receive a newinstruction with each cycle. Pipelines sometimes stall and/or may needto be flushed and restarted, which may cause the pipeline to beunavailable for one or more cycles. Furthermore, the fact that aninstruction is in one of registers 204 does not mean that it isavailable to execute. There may be other preceding instructions whichprogram logic requires execute first (although in many cases it ispossible to execute multiple instructions concurrently or out ofsequence without corrupting the program logic).

Instructions within each executing thread are grouped in groups ofnon-dependent sequential instructions. When instructions are loaded intobuffers 204 from L1 I-Cache 201, dependency check unit 308 automaticallyanalyzes the instruction stream to determine which instructions can beexecuted concurrently or out of sequence. The dependency check unitdivides the instruction stream into groups of sequential instructions byplacing a stop bit at the last instruction of each group. Within anygroup, the instructions have no dependencies and can safely be executedout of sequence. Specifically, with respect to each instruction, thedependency check unit determines whether there is some dependency uponcompletion of a previous instruction in the same group. Dependencydepends on the type of instruction and the operand(s). Where twodifferent instructions reference the same operand, this often, but notalways, creates a dependency. E.g., if both instructions use the sameoperand as a source for data which is read or input to some pipeline, nodependency exists. But if a first instruction writes to an operandlocation and a second instruction reads from the same location, thesecond instruction is clearly dependent on the first, and can not beexecuted before or simultaneously with the first instruction. If adependency is found within the same group, the dependency check unitplaces a stop bit in the immediately preceding instruction, so that theinstruction being analyzed will be part of a separate group. Whendecode/dispatch unit 303 selects instructions for dispatch, it won'tproceed beyond a group (as indicated by a stop bit) until all theprevious instructions have been dispatched.

Because the T0 thread registers are reserved for use by threads of thespecial class (while the remaining register sets can be used by any typeof thread), most of the time there is no active T0 thread executing inthe processor, and the T0 register sets spend most of their time idle.When there is no active T0 thread, decode/dispatch unit 303 willgenerally select instructions from the T1 thread for dispatch, andsometimes select T2 thread instructions where hardware resource isavailable and no T1 thread instruction can be dispatched to fill it.When a T0 thread is active, it takes priority over the others, and T1and T2 threads are consequently dispatched with significantly lessfrequency. In this mode, the T2 thread, being the lowest priority, mightrarely or never get its instructions dispatched, but its state remainsin its register set, ready to resume more frequent execution when the T0thread relinquishes the processor.

While various CPU components have been described and shown at a highlevel, it should be understood that the CPU of the preferred embodimentcontains many other components not shown, which are not essential to anunderstanding of the present invention. Furthermore, it will beunderstood that the CPU of FIG. 3 is simply one example of a CPUarchitecture, and that many variations could exist in the number, typeand arrangement of components within CPU 101, that components not shownmay exist in addition to those depicted, and that not all componentsdepicted might be present in a CPU design. For example, the number andconfiguration of buffers and caches may vary; the number and function ofexecution unit pipelines may vary; registers may be configured indifferent arrays and sets; dedicated floating point hardware may or maynot be present; etc. Furthermore, CPU 101 may have a simple or complexinstruction set.

In the preferred embodiment, the T0 threads are restricted to using aportion of certain caches. The reason for this restriction is that a T0thread, operating at a high priority, will tend to flush the cache andreplace its contents with its own data. After brief execution, the T1and T2 threads then must refill the cache with their data. However, theT0 thread typically doesn't need as large a cache space as the otherthreads. Its working set is typically smaller. If unrestricted, it willcause the least recently used cache lines to be evicted; usually, theseare the lines associated with other threads. But if the T0 thread isrestricted to some portion of the cache (which is still large enough toprovide an adequate working set), it ends up evicting its own cachelines, which it no longer needs, rather than older cache lines whichwill be needed in the future by the other threads.

Generally, the effect of flushing caused by a high priority thread isgreater at the higher cache levels. Because the T0 thread typically doesnot require a lot of data, the larger, lower level caches are unlikelyto be significantly flushed even if unrestricted. Therefore, in thepreferred embodiment, only the L1 caches are restricted, i.e., the T0threads are restricted to a portion of the L1 caches only, and are notrestricted to any portion of the L2 or L3 caches. It would alternativelybe possible to restrict the T0 threads to some appropriate portion theL2 and/or L3 caches.

FIG. 4 is a representation of the general structure of a cache includingassociated accessing mechanisms, according to the preferred embodiment.FIG. 4 could represent any of either L1 I-cache 201, L1 D-cache 202, L2cache 203, or L3 cache 204, although in the preferred embodiment theeviction selection logic 410 is different in the L1 caches, as explainedfurther herein.

As is known in the art, a cache is accessed by decoding anidentification of an associativity set from selective address bits (orin some cases, additional bits, such as a thread identifier bit), andcomparing the addresses of the cache lines in the associativity set withthe desired data address. For example, where there are 1K associativitysets in a cache, 10 bits are needed to specify a particularassociativity set from among the 1K. Ideally, these 10 bits aredetermined so that each associativity set has an equal probability ofbeing accessed. In the preferred embodiment, L1 caches 201, 202 areaddressed using effective addresses, while L2 cache 203 and L3 cache 204are addressed using real addresses. Therefore, in order to access L2 orL3 caches, an effective address generated by the processor is firsttranslated to a real address by address translation hardware 321.Alternatively, as is known in some computer system designs, it would bepossible to access all cache levels using real addresses, or to accesssome or all of the caches below L1 using virtual or effective addresses,without translation.

Referring to FIG. 4, a cache comprises a cache data table 401 and acache index 402. The data table 401 contains multiple cache lines ofdata 403 grouped in associativity sets 404. In the preferred embodiment,each cache line 403 is the same size in all of the caches, although thisneed not be the case. A cache line is the smallest unit of data whichcan be brought into the cache or evicted from the cache. A typical cacheline size is 128 bytes, although this could vary. The number of cachelines in each associativity set 404 typically varies, this number beinggreater for lower level caches. For example, L1 caches might containeight cache lines in each associativity set, while L2 and L3 caches havesixteen lines in each associativity set, it being understood that theseare merely representative parameters which may vary.

Index 402 contains multiple rows 405 of index entries 406, each row 405corresponding to a respective associativity set 404 of data table 401and containing a number of index entries 406 equal to the number ofcache lines 403 in the associativity set 404. Each index entry 406contains at least a portion of an address 421 of a corresponding cacheline 403, certain control bits 422, and a thread identifier 423. Addressportion 421 could be a portion of a real or effective address, dependingon which is used to access the cache. Control bits 422 may include, butare not necessarily limited to: a modified bit; an invalid bit; one oremore bits for selecting a cache line to be evicted where necessary, suchas least-recently-used (LRU) bits; one or more bits used as semaphores;locks or similar mechanisms for maintaining cache coherency; etc., asare known in the art.

A cache line is referenced by selecting a row 405 of index 404corresponding to some function of a portion of the address 420 of thedesired data (herein called the reference address), using selector logic407. This function is selected to achieve an approximately equaldistribution of data references among the various associativity sets.Often, this function is a direct decode of the N bits of referenceaddress at bit positions immediately above the M lowest bitscorresponding to the cache line size. For example, for a cache line sizeof 128, or 2⁷, M is 7). N depends on the number of associativity sets inthe cache, and is sufficiently large to select any associativity set.Generally, this means that N is the base 2 log of the number ofassociativity sets. I.e., for a cache having 1024 associativity sets, Nis 10. However, more complex hashing functions could alternatively beused, which could include bits outside an address, such as a threadidentifier. Furthermore, the functions need not be the same for allcaches; a direct decode may be used for one cache while a more complexhashing function is used for another. The reference address containsmore than (N+M) bits, so that multiple addresses map to the sameassociativity set.

The selective bits of reference address are input to selector logic 407,which generates a selection signal to select a single row 405 of thecache index 402. The address 411 in each respective index entry 406 ofthe selected row 405 is then compared with the reference address 420 bycomparator logic 409. In fact, it is usually only necessary to comparesome portion of the reference address which was not used by selector407, since the bits used by selector 407 inherently compare by virtue ofthe row selection. If there is a match, comparator logic 409 outputs aselection signal corresponding to the matching one of the index entries.Selector logic 408 selects an associativity set 404 of cache lines 403using the same reference address bits used by selector 407, and theoutput of comparator 409 selects a single one of the cache lines 403within the selected associativity set. The selected line is the dataoutput 413 of the cache. The various selection lines output bycomparator 409 are logically ORed by OR gate 412 to produce a cache hitsignal. I.e., if the output of OR gate 412 is active, a cache hit hasbeen detected and the data output 413 can be considered valid; if theoutput of OR gate 412 is not active, there has been a cache miss.

Although selectors 407 and 408 are shown in FIG. 4 as separate entities,it will be observed that they perform identical function. Depending onthe chip design, these may in fact be a single selector, having outputswhich simultaneously select both the index row 405 in the index 402 andthe associativity set 404 in the cache data table 401.

When a data reference to the cache produces a cache miss (the requireddata is not in the cache), it is generally necessary to load therequired data from a lower level cache or from main memory 102.Depending on the system design, it is sometimes also possible to loaddata to cache prospectively, although this is usually done only forlower level caches, and not for the L1 cache. When a new cache line isto be loaded into cache, it is necessary to select an existing line tobe replaced (“evicted” from the cache). Selection is performed byeviction selection logic 410.

A line being evicted must necessarily belong to the same associativityset as the line which will replace it. A line is selected for evictionby selecting the appropriate row 405 of cache index 402 using selector407, and inputting selective control bits 422 and thread identifier 423of each entry 406 to eviction selection logic 410. Eviction selectionlogic 410 outputs a selection signal, which is used to select a cacheline 403 of an associativity set 404 for writing out of cache data table401, or for loading a new cache line into the cache data table from loadbus 413. When loading a new cache line, the output of eviction selectionlogic 410 also drives write selector 411, which selects an entry 406 ofa row 405 of cache index for writing data from load bus 413, address420, or other control information applicable to the newly loaded cacheline.

In the preferred embodiment, for at least one of the caches, evictionlogic 410 effectively limits the number of cache lines in eachassociativity set which may be filled by the T0 thread. This number isreferred to as the T0 limit (T0L), and it is necessarily less than thenumber of cache lines in each associativity set. An external signal (T0)indicates to the eviction logic whether the thread which requested thenew cache line is the T0 thread. If not, the eviction logic selects anycache line of the associativity set for eviction, according to anyappropriate eviction algorithm. Numerous such algorithms exist inconventional art, and any such algorithm, now known or hereafterdeveloped, could be used. If the T0 thread requested the new cache line,and if there are already T0L cache lines in the associativity set whichare filled by thread T0, then the eviction logic is constrained to evictone of the cache lines which were filled by thread T0. In this manner,the number of lines in the cache which were filled by thread T0 can notexceed T0L.

Eviction logic 410 can easily implement this restriction by altering aneviction priority under certain circumstances. Normally, eviction logic410 derives a priority for each cache line of a selected associativityset, and selects a low priority line for eviction. The priority isnormally derived from certain bits of control field 422, such as LRUbits which indicate recency of reference, invalid bit, etc., accordingto any of various known techniques or techniques hereafter developed.This priority (Std_Priority) is modified if the new cache line is beingrefilled by thread T0 and there are already T0L or more cache lines inthe associativity set which were filled by T0. The modified priorityassigned to each cache line may be expressed as follows: IF (ΣCache_Line_T0(i) + New_Line_T0 > T0L) AND (Cache_Line_T0(S) = 0) THENPriority_Cache_Line(S) = MAX ELSE Priority_Cache_Line(S) =Std_Priority_Cache_Line(S)where New_Line_T0 is 1 if the thread which is filling the new cache lineis a T0 thread and zero otherwise; Cache_Line_L0(i) is 1 if cache line iwas filled by thread T0, and 0 otherwise (the summation being madeacross all cache lines i in the associativity set); MAX is a highestpossible priority; Priority_Cache_Line(S) is the modified priority ofcache line S, and Std_Priority_Cache_Line(S) is the unmodified priorityof cache line S based on LRU, invalid and/or other control bits.Preferably, MAX is a special high priority level which is higher thanany other level, although it could alternatively be the highest levelbased on LRU (i.e., a level equivalent to the most recently accessedcache line). It would alternatively be possible to modify the aboveexpression by adding the additional condition that cache line S must bevalid to receive the MAX priority. This additional condition might allowan invalid cache line to be replaced by a cache line requested by threadT0, and therefore in limited circumstances the cache lines filled bythread T0 might exceed the T0L limit.

As described above, threads of a special class execute using the T0register set in processor 101. It will be observed that there is nothingin the processor itself which would prevent any type of thread fromexecuting in the T0 register set. The processor relies on the operatingsystem to identify threads of the special class which should execute inthe T0 register set. The operating system's dispatcher separatelyrecords thread type and indicates to the processor the register set inwhich a dispatched thread should execute.

FIG. 5 is a conceptual illustration showing certain major softwarecomponents of memory 102 and the division of hardware and softwarefunction at different levels of abstraction in computer system 100. Asis well known, a computer system is a sequential state machine whichperforms processes. These processes can be represented at varying levelsof abstraction. At a high level of abstraction, a user specifies aprocess and input, and receives an output. As one progresses to lowerlevels, one finds that these processes are sequences of instructions insome programming language, which continuing lower are translated intolower level instruction sequences, and pass through operating systemcode and ultimately to data bits which get put in machine registers toforce certain actions. At a very low level, changing electricalpotentials cause various transistors to turn on and off. In FIG. 5, the“higher” levels of abstraction are represented toward the top of thefigure, while lower levels are represented toward the bottom.

Hardware level 501 shown in FIG. 5 represents the physical processors,memory, buses, and other components which cause instructions to beexecuted. As used herein, hardware level 501 represents the collectionof physical devices (as opposed to data stored in devices) shown inFIGS. 1-4, including other hardware not shown in FIGS. 1-4.

Immediately above hardware is low-level operating system level 502,which in some operating systems is called a “kernel”. In a physicalsense, the operating system is code, i.e., it is data in the form ofinstructions which is stored in various memory locations and executed onone or more processors to perform the required function. The low-leveloperating system provides certain base operating system function neededto share system resources, allocate memory, enforce security, and soforth. Among the functions provided by low-level operating system 502 isa thread dispatching function 503. Dispatcher 503 dispatches threadswaiting to be executed to processors for execution, as explained ingreater detail herein. Associated with dispatcher 503 are a ready queuedata structure 504 containing threads ready to execute and waiting fordispatch by dispatcher 503; a wait queue data structure 505 containinginstantiated threads which are not immediately ready to execute, i.e.,that are waiting for some event (such as completion of an I/O operation)to occur before they can be dispatched to a processor for execution; andexecution state data structures (represented as entities 506A-506D, andreferred to herein generically as feature 506) which records the threadswhich are currently executing in each respective processor 101A-101D.

Above the level of low-level operating system 502 are various userapplications 510-512 and user data objects 513-518. The system mayoptionally include additional higher-level operating system functions509. In general, higher-level operating system functions 509 provideadditional capabilities and functions to users who wish to access them,but a user process may directly access the low-level operating system502 for execution.

Thread dispatching depends on the state and priority of the threads. Atany instant in time, a thread is in one of several states. For example,a thread may be in an executing state, in which it has been dispatchedto a processor for execution, an event wait state, in which it can notexecute until the occurrence of some external event and is thereforewaiting for the event to occur, or a ready state, in which the thread isready to execute and is waiting only for an available processor.Depending on the operating system, additional states or refinements ofthe above states may be defined. Additionally, a priority of executionis typically associated with each thread. Any of various priorityassignment schemes known in the art or hereafter developed may be used.The priority is typically assigned by the user, system administrator, oroperating system itself. E.g., the priority of a user applicationprocess is often a default priority for user processes specified by theoperating system, subject to override by the user. The priority may befixed for the duration of the thread's existence, or may be adjustabledepending on various factors, such as the length of time the thread hasbeen waiting in the ready queue.

In the preferred embodiment, among the state information carried by eachthread or group of threads is whether the thread or group is a member ofa “special class” of threads. This information could be carried as asingle bit, although it would alternatively be possible to have multiplespecial or other classes of threads requiring multiple bits of stateinformation.

Membership in the special class is preferably designated by any ofvarious alternative techniques. For example, the operating system couldautomatically assign certain kernel threads to the special class basedon same thread attribute or combination of thread attributes. It mayeven be possible for the operating system to dynamically assign threadsto the special class based on observed behavior, such as the quantity ofinstructions or time it executed when previously dispatched. Membershipmay be a fixed attribute of the code, which is manually specified by auser and assigned at compilation time, build time, load time, or othertime in advance of instruction execution. Since it may be undesirable toallow any arbitrary user to specify that his code or portions thereofexecute as a special class thread, special tools, available only toauthorized persons, are preferably used to designate membership in thespecial class.

In general, it is expected that special class threads will be low-leveloperating system functions rather than user application code. Threadsnot designated as belonging to the special class are considered “generalclass” threads. In FIG. 5, special class threads are represented as“S”N, and general class threads are represented as “G”N, where N is aninteger.

Thread ready queue structure 504 and thread wait queue structure 505identify a variable number of threads, and data therein may bestructured according to any appropriate conventional form. These queuestructures are represented in FIG. 5 as respective single lists orqueues, although in fact either may comprise multiple lists separatedaccording to priority or other considerations. As represented in FIG. 5,the threads on the ready queue 504 are typically general class threads,since special class threads tend to be dispatched immediately; the waitqueue 505 typically contains a mixture of special class threads andgeneral class threads.

A thread remains on wait queue 505 until some event on which it waswaiting occurs. At that point, the thread is transferred to ready queue504. Dispatcher 503 selects threads for dispatching from ready queue504. Typically, threads are selected according to priority, and amongthreads of equal priority, the thread which has been in the ready queuelongest is selected first. However, other considerations could be used.For example, in some systems such as non-uniform memory architecture(NUMA) systems or systems which are logically partitioned, separatequeues might be maintained for different processors or groups ofprocessors within the system and/or threads dispatched based on someform of association with a processor or group of processors.Furthermore, although the present system has been described as having asingle operating system and dispatcher which dispatches threads of thespecial class to the T0 register set, a computer system might bepartitioned into multiple logical partitions, each containing its ownrespective operating system and dispatcher, and in this case adispatcher which selects threads of the special class for dispatch tothe T0 register set might be present in one, some or all of the logicalpartitions.

Execution state data structures 506 record the threads executing in eachregister set of each processor. For example, as shown in FIG. 5, datastructure 506A, which is associated with processor 101A, records thread“S4” in the T0 thread register set, thread “G3” in the T1 register set,and thread “G8” in the T2 register set. Generally, the T1 and T2register sets are busy, and so general class threads will be waiting onready queue 504. However, in general most of the special class threadswill be waiting on wait queue 505, and only a few will be executing atany given moment. Typically, the number of executing special classthreads will not exceed the number of processors, i.e., the number of T0register sets. As shown in FIG. 5, T0 register sets for processors 101Band 101D (as recorded in execution state data structures 506B and 506D,respectively) are idle.

Threads of the general class are dispatched from ready queue 504 only toan available T1 or T2 register set, and never to a T0 register set, evenif one is idle. When a T1 or T2 register set in any processor becomesavailable (either because an executing thread relinquishes theprocessor, or is pre-empted), dispatcher 503 chooses a thread from readyqueue 504 and dispatches it to the processor. If a T1 register setbecomes available, the dispatcher may cause the processor to switch theexisting T2 thread to the T1 register set, and dispatch the enqueuedthread to the T2 register set. Generally, a thread dispatched from theready queue to a T1 or T2 register set is a general class thread,although in rare cases it might be a special class thread, as discussedbelow.

When an event occurs which causes a thread waiting on the wait queue 505to become ready to execute, the thread is transferred to ready queue504. If the thread is a special class thread, it is usually a highpriority thread and goes to the head of the ready queue for immediatedispatch. A special class thread is dispatched to a T0 register set, ifthere is one available. Usually, there will be at least one T0 registerset available, to which the special class thread is immediatelydispatched. In rare circumstances, all of the T0 register sets may bebusy. If this happens, the special thread may be dispatched to a T1register set (or even a T2 register set). Because the special classthread has a high priority, it will typically preempt an existinggeneral class thread, and be dispatched immediately to a T1 register setin these circumstances. If a special class thread is dispatched to a T1register set, it executes in the T1 set just as any general class threadwould, and is not restricted to some subset of the cache lines in the L1caches (as a thread executing in the T0 register set would be). Anoperating system tuning parameter controls whether a special thread maybe dispatched to the T1 and T2 logical processors.

Various software entities are represented in FIG. 5 as being separateentities or contained within other entities. However, it will beunderstood that this representation is for illustrative purposes only,and that particular modules or data entities could be separate entities,or part of a common module or package of modules. Furthermore, althougha certain number and type of software entities are shown in theconceptual representation of FIG. 5, it will be understood that theactual number of such entities may vary, and in particular, that in acomplex multi-processor system environment, the number and complexity ofsuch entities is typically much larger.

While the software components of FIG. 5 are shown conceptually asresiding in memory 202, it will be understood that in general the memoryof a computer system will be too small to hold all programs and datasimultaneously, and that information is typically stored in data storagedevices 125-127, comprising one or more mass storage devices such asrotating magnetic disk drives, and that the information is paged intomemory by the operating system as required. Furthermore, it will beunderstood that the conceptual representation of FIG. 5 is not meant toimply any particular memory organizational model, and that system 100might employ a single address space virtual memory, or might employmultiple virtual address spaces which overlap.

The embodiment of the present invention herein described, with one highpriority but cache limited hardware thread and two hardware regularthreads, can best be compared to a processor with three regular threads.For the purpose of comparison, the situation in which there are two orthree CPU compute-intensive application threads and several I/Ointensive threads is considered. The I/O intensive threads could bedoing asynchronous I/O in behalf of the compute-intensive applicationsand/or be performing utility functions such as copying files from onedevice to another.

In the case of a three-threaded conventional processor without thedesignation of a special register set, in order to dispatch a highpriority I/O thread, the operating system performs a “context switch,”saving the state of one of the three running compute-intensive threadsand then restoring the state of the I/O thread. While it is running, ifthe I/O thread copies a block of data from one memory buffer to another,it will flush out much if not all the compute threads' data from cache.The two still-running compute intensive applications are significantlyslowed down by I/O thread's sweep of the cache. When the I/O threadenters a wait state, the operating system performs another contextswitch, saving the state of the I/O thread and restoring the state ofthe ready-to-run compute intensive thread. Now all three of the computethreads are running, but sluggishly, since they all repeatedlyexperience cache misses and these cache misses cause instructions totake more clock cycles to execute.

By comparison, the preferred embodiment avoids saving/restoring ofcompute-intensive thread context due to dispatching an I/O thread. Italso avoids the cache misses inflicted by the I/O thread upon thecompute threads.

In a system busy with many concurrent I/O threads, there will be manycontext changes involving the I/O threads, as they become dispatchableafter an I/O interrupt. In the embodiment of the invention, they willpreferably exploit the high priority hardware thread rather than thegeneral purpose threads.

Where there are two or three compute intensive application threads, withthe preferred embodiment, none experiences interruption and contextswitches to service the I/O intensive threads. Where there are threecompute-intensive application threads, if two or more share the sameapplication priority, there are however, interruptions and contextswitches among them by the operating system as it time-shares systemresources among these application threads.

These problems are generally more acute in the case of a conventionalprocessor supporting only two threads.

In the preferred embodiment, eviction selection logic in the cacheoperates to limit the number of cache lines which can be filled by a T0thread by preferentially evicting T0-filled-lines when the number ofT0-filled-lines is at a predetermined limit T0L. This embodiment has theadvantage of simplicity of implementation, and is flexible enough toallow all cache lines to be used by non-T0 threads when there is noactive T0 thread. However, there are various alternative ways to limitthe number of cache lines used by a T0 thread. For example, a fixedsubset of lines in each associativity set could be reserved for the T0threads, which could not use any other cache lines. Such a subset couldbe reserved exclusively for T0 threads, or could be used by otherthreads when there is no active T0 thread. Alternatively, the T0 threadmight have its own separate cache at the same level, which is notaccessible to the other threads. The isolation provided by a separatecache may, in some circumstances, provide better performancenotwithstanding the inability of the T1 or T2 threads to use the extracache lines dedicated to the T0 thread.

In the preferred embodiment, threads of a special class are dispatchedto a dedicated high priority register set, and are constrained toutilize only a subset of cache lines in a cache. However, it wouldalternatively be possible to utilize only one of these features,independently of the other. For example, a special high priorityregister set might be set aside for the special class threads, withoutplacing any restrictions on the number of cache lines it could utilize.Alternatively, it would be possible to provide a restriction on thenumber of cache lines which can be filled by threads of a certain class,without providing a special register set for such threads.

In the preferred embodiment, threads of a special class utilize adedicated, high priority set of registers which are unavailable togeneral class threads, and are constrained to utilize only a subset ofcache lines in a cache. However, it would alternatively be possible toprovide other forms of asymmetric hardware resource support for threadsof the special class. For example, in general it is expected thatthreads of the special class do not perform extensive floating point orvector operations. If a processor contains vector or special floatingpoint pipelines and other logic for efficiently performing floatingpoint operations, including vector and floating point registers tosupporting such operations, it may be possible in some processor designsto omit such registers in the T0 register set. Such an omission wouldachieve a small reduction in hardware resources required for supportingthe T0 register set, although it might require that the compiler be ableto compile T0-eligible threads in a special mode which uses no floatingpoint instructions. Additional hardware reductions or modifications forthe T0 register set may be possible.

As described herein, a multithreaded processor supports the concurrentexecution of N threads, where N is preferably 3. However, the choice ofN could depend on many design factors, and could be a number other than3. As processors become more capable, it is possible that designconsiderations will favor a larger number of threads. Furthermore,although in the preferred embodiment a single register set is reservedfor threads of the special class, the number of register sets reservedfor threads of the special class could conceivably be higher than one.

In general, the software routines executed to implement certainillustrated embodiments of the invention, whether implemented as part ofan operating system kernel or otherwise, are referred to herein as“programs” or “computer programs”. The programs typically compriseinstructions which, when read and executed by one or more processors inthe devices or systems in a computer system consistent with theinvention, cause those devices or systems to perform the steps necessaryto execute steps or generate elements embodying the various aspects ofthe present invention. Moreover, while the invention has and hereinafterwill be described in the context of fully functioning computer systems,the various embodiments of the invention are capable of beingdistributed as a program product in a variety of forms, and theinvention applies equally regardless of the particular type ofsignal-bearing media used to actually carry out the distribution.Examples of signal-bearing media include, but are not limited to,volatile and non-volatile memory devices, floppy disks, hard-diskdrives, CD-ROM's, DVD's, magnetic tape, and so forth. Furthermore, theinvention applies to any form of signal-bearing media regardless ofwhether data is exchanged from one form of signal-bearing media toanother over a transmission network, including a wireless network.Examples of signal-bearing media are illustrated in FIG. 1 as systemmemory 102, and as data storage devices 125-127.

Although a specific embodiment of the invention has been disclosed alongwith certain alternatives, it will be recognized by those skilled in theart that additional variations in form and detail may be made within thescope of the following claims:

1. A digital data processing device, comprising: a processor, saidprocessor containing a plurality of register sets for supporting themaintenance of thread state and concurrent execution of a plurality ofthreads, each register set of said plurality of register setscorresponding to a respective thread; a memory for storing instructionsexecutable by said processor; an operating system embodied as aplurality of processor-executable instructions, said operating systemincluding a dispatcher for selecting threads for dispatch to saidprocessor; wherein said threads include a plurality of threads of aspecial class and a plurality of threads not belonging to said specialclass, and wherein at least one said register set is reserved for saidthreads of a special class.
 2. The digital data processing device ofclaim 1, wherein said threads of a special class comprise high-priorityI/O bound threads.
 3. The digital data processing device of claim 1,wherein a respective execution priority is associated with each saidregister set of said plurality of register sets; wherein said processorcomprises an instruction unit which selects instructions for executionfrom among said plurality of register sets, said instruction unitpreferentially selecting instructions according to said respectiveexecution priority of each said register set.
 4. The digital dataprocessing device of claim 3, wherein said register set reserved forsaid threads of a special class has a higher execution priority than anyregister set of said plurality of register sets which is not reservedfor said threads of a special class.
 5. The digital data processingdevice of claim 1, wherein said dispatcher dispatches threads forexecution in dispatcher-specified register sets; and wherein saiddispatcher dispatches said threads not belonging to said special classonly to register sets other than said at least one register set reservedfor said threads of a special class.
 6. The digital data processingdevice of claim 1, further comprising: at least one cache fortemporarily storing selective contents of said memory for use by threadsexecuting on said processor; wherein said threads of a special classexecuting on said processor are prevented from filling more than aportion of said cache, said portion being less that all of said cache.7. A digital data processing device, comprising: a processor, saidprocessor containing a plurality of register sets for supporting themaintenance of thread state and concurrent execution of a plurality ofthreads, each register set of said plurality of register setscorresponding to a respective thread; a memory for storing instructionsexecutable by said processor; a cache for temporarily storing selectivecontents of said memory for use by threads executing on said processor;and wherein said threads include a plurality of threads of a specialclass and a plurality of threads not belonging to said special class;and wherein said threads of a special class executing on said processorare prevented from filling more than a portion of said cache, saidportion being less that all of said cache.
 8. The digital dataprocessing device of claim 7, wherein said threads of a special classcomprise high-priority I/O bound threads.
 9. The digital data processingdevice of claim 7, wherein said cache comprises a plurality of cachelines, each line being equally available to said threads of a specialclass; and wherein said data processing device further comprises a cachefilling facility which restricts the number of said cache lines filledby said threads of a special class.
 10. The digital data processingdevice of claim 9, wherein said cache comprises a plurality ofassociativity sets, each associativity set containing N cache lines,where N is an integer greater than one; and wherein said cache fillingfacility comprises eviction logic which selects cache lines for evictionfrom said cache, said eviction logic preferentially selecting a linefilled by a thread of said special class for eviction from anassociativity set when the total number of cache lines in theassociativity set filled by threads of the special class equal or exceeda predetermined value less than N.
 11. The digital data processingdevice of claim 7, wherein said cache is divided into a plurality ofdiscrete portions, at least one portion being allocated to threads ofsaid special class and at least one portion being allocated to threadsnot belonging to said special class.
 12. The digital data processingdevice of claim 7, wherein said at least one cache comprises at leastone cache at a first level and at least one cache at a second level,said first level being closer to said processor than said second level,and wherein said threads of a special class executing on said processorare prevented from filling more than a portion of said at least onecache at said first level, said portion being less that all of said atleast one cache at said first level, but are not prevented from fillingmore than a portion of said at least one cache at said second level. 13.A computer program product for supporting dispatching functions,comprising: a dispatcher for a multithreaded processor implemented as aplurality of computer-executable instructions recorded on signal-bearingmedia, wherein said instructions, when executed by at least one computersystem, cause the at least one computer system to select threads fordispatch to said multi-threaded processor wherein said multi-threadedprocessor contains a plurality of register sets for supporting themaintenance of thread state and concurrent execution of a plurality ofthreads, each register set of said plurality of register setscorresponding to a respective thread; wherein said threads include aplurality of threads of a special class and a plurality of threads notbelonging to said special class, and wherein at least one said registerset is reserved for said threads of a special class; and wherein saiddispatcher selects a register set from among said plurality of registerssets of said multi-threaded processor to which a thread is dispatchedaccording to whether the thread being dispatched is a thread of saidspecial class.
 14. The computer program product of claim 13, whereinsaid threads of a special class comprise high-priority I/O boundthreads.
 15. A digital data processor chip for use in a digital dataprocessing system, comprising: a processor core, said processor corecontaining a plurality of register sets for supporting the maintenanceof thread state and concurrent execution of a plurality of threads, eachregister set of said plurality of register sets corresponding to arespective thread; a cache use by threads executing on said processorcore, said cache temporarily storing selective contents of a memory ofsaid digital data processing system for use by threads executing on saidprocessor; and wherein said threads include a plurality of threads of aspecial class and a plurality of threads not belonging to said specialclass; and wherein said threads of a special class executing on saidprocessor are prevented from filling more than a portion of said cache,said portion being less that all of said cache.
 16. The digital dataprocessor chip of claim 15, wherein said cache comprises a plurality ofcache lines, each line being equally available to said threads of aspecial class; and wherein said data processing device further comprisesa cache filling facility which restricts the number of said cache linesfilled by said threads of a special class.
 17. The digital dataprocessor chip of claim 16, wherein said cache comprises a plurality ofassociativity sets, each associativity set containing N cache lines,where N is an integer greater than one; and wherein said cache fillingfacility comprises eviction logic which selects cache lines for evictionfrom said cache, said eviction logic preferentially selecting a linefilled by a thread of said special class for eviction from anassociativity set when the total number of cache lines in theassociativity set filled by threads of the special class equal or exceeda predetermined value less than N.
 18. The digital data processor chipof claim 15, wherein said cache is divided into a plurality of discreteportions, at least one portion being allocated to threads of saidspecial class and at least one portion being allocated to threads notbelonging to said special class.
 19. The digital data processor chip ofclaim 15, wherein said processor chip comprises at least one cache at afirst level and at least one cache at a second level, said first levelbeing closer to said processor core than said second level, and whereinsaid threads of a special class executing on said processor core areprevented from filling more than a portion of said at least one cache atsaid first level, said portion being less that all of said at least onecache at said first level, but are not prevented from filling more thana portion of said at least one cache at said second level.
 20. Thedigital data processor chip of claim 15, wherein at least one saidregister set is reserved for said threads of a special class.